Applications with strict real-time requirements, such as video conferencing and gaming specify an acceptable latency (for example, 100 milliseconds to 300 milliseconds) for a video encoder. Pursuant to an exemplary scenario, a video rate control (RC) mechanism of the video encoder may utilize a hypothetical model of a decoder, such as video buffer verifier (VBV), specified by video compression standards, such as Moving Pictures Expert Group (MPEG)-4 and International Telecommunication Union Standardization Sector (ITU-T), to achieve an adherence to the latency specifications. The RC mechanism may be configured to skip encoding of pictures corresponding to the video data upon an occupancy of the VBV approaching fullness during encoding. The pictures may be skipped during encoding until the occupancy of the VBV reduces below a maximum allowable occupancy, so as to ensure adherence to the latency specification. Pursuant to an exemplary scenario, in order to reduce the number of pictures skipped during encoding, a limit is placed on a maximum size of encoded picture. However, such a limit may affect a picture quality when a picture prediction is not effective and a residual energy is large.